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PEER RATINGS: SCORING STRATEGY DEVELOPMENT AND RELIABILITY 
DEMONSTRATION ON AIR FORCE BASIC TRAINEES 

I. INTRODUCTION 

Personnel selection research in the U.S. military has typically 
employed paper-and-pencil tests of various types to predict perform- 
ance in training and on the job. However, another strategy for 
predicting training and job performance in the military has been to 
gather peer ratings of potential for effectiveness. Peer ratings have 
enjoyed considerable success in predicting a varietv of effectiveness 
criteria in the military. Within the Air Force, 'peer ratings have 
correlated successfully with pilot training success (Flyer & Bigbee. 
1955), instructor pilot job performance (Swanson & Johnson, 1975). 
Ufficer Training School graduate performance (Tupes, 1957). and 
performance as cadets, officer candidates, and officers (Tupes & 
Kaplan, 1961). Annotations of these studies and the considerable peer 
rating research conducted in the Army, Navy, and Marine Corps can be 
found in Lammlein and Bormatn (1979). 

The objective of the present study was to evaluate the psychometric 
Characteristics of peer ratings among Air Force basic trainees. If 
the psychometric properties of these peer ratings are satisfactory, 
then future research should access the utility of peer evaluations by 
themselves and in combination with test scores, for predictino success 
in advanced training and subsequent job performance. 

Special Problems in Scoring Nominations 

Raters in the present study were asked to identify the five best or 
most effective trainees in their flight (Ns = 29-58) on each of 
several categories (e.g.. Try Hard, Cooperative). They were also 
asked to identify the five worst or least effective trainees in their 
flight on each of these categories. The main problem this kind of 
aata presents is how to score nominations when different groups 
contain different numbers of persons and scores must be compared 
across groups. Kane and Lawler (1978) and Willingham (1959a) present 
detailed descriptions of this problem. m'"c>'i. 

?Kiln?.c difficulties Stem primarily from two sources. The most 
obvious problem is that members of large groups have a greater 
potential of receiving many nominations (negative or positive) than do 
members of smaller groups. This creates larger variances in raw 
scores for larger groups and results in non -comparable nomination raw 
scores across different-sized groups. The difficulty has typically 
been addressed by dividing the sum of the nominations for each grouo 
member by the total number of nominators in the group; however, 
Willingham (1959a) argued that this correction does not in itself 
fu "°r" /5'^°" groups. As evidence. Willingham demonstrated 
that the adjusted scores correlate highly (and negatively with the 



size of the groups, and therefore this adjustment provides an over- 
correction, (i.e., corrected scores in larger groups now tend to have 
smaller variances). Willinghaiti offered another correction formula 
fIrlSccessfully equates the variances of adjusted scores for 
different-sized groups, and thus aids in creating comparable scores 
across groups. 

Willingham's formula does not address the problem of differing 
kurtosis (peakedness of the distribution for different-sized groups. 
Lalier groups tend to have more peaked distributions than do smaller 
groups because a large proportion of the members of large groups 
typically receive zero or._.a small number of nominations. This too 
creates inequities in scorina nominations for groups of different 
sizS! and Willingham (1959a? suggested that similar distributions 
across groups could be obtained by asking group members to provide 
different numbers of nominations according to their own group s size. 

presen tiSaTormul a to compute the number of nominations required 
for differing group sizes. 

For purposes of the present project, it was not feasible to employ 

Sese Tggelt?ons directly. First, "ilV"^'"^ ^lr;hf rnstruSn 
assumes that persons can nominate themselves, while the instructions 
in tJil peer nomination administration asked that persons not nominate 
themselves! Also, his formula requires that each group member 
Sated provides nominations, and this requirement was not met in 
the present study.l Willingham's suggestion for handling differing 
levels of kurtosis could not be employed, either, because data had 
already been collected on a form requiring five PO/^t^^^ f^^^^ 
negative nominations for all groups. Accordingly, for the present 
study a norming strategy was devised in which flights with similar 
numbers of nominees were pooled together, and then standard zed 
nomination scores were developed for each grouping. These procedures 
are discussed in the Method section. 

II. METHOD 

Subjects 

Subiects in the present research were more than 24,000 students 
assigned ti Air Force Basic Training at Lackland AFB Texas. The mean 
age of the trainee sample was 20.29, with a standard deviation of 2.17. 



iTypically, flights (student groups) contained more nominees 
than nominators because trainees were instructed to nominate anyone in 
Seir fTight^ including those who for various reasons were absent from 
the rating schedule. 



Data Collection 



The peer nominations were gathered by staff members of the basic 
training course. In general, data collection was accomplished for 
each flight separately. Student raters were given a roster of their 
flight members and, as mentioned, were instructed to nominate the five 
best and five worst trainees on each of eight rating dimensions. They 
were told not to nominate themselves. 



Method for Scoring the Nominations 

A special norming strategy was developed for adjusting raw nomination 
scores so that they would be comparable across the different flights. 
Specifically, the strategy combined empirical norming and a procedure 
to equate the expected variances of adjusted scores for different 
flights. Flights with approximately equal numbers of nominees were 
grouped together (the number of nominators varied to a degree across 
these groups^), and this brought together flights with very similar 
levels of kurtosis, because the expected kurtosis varies according to 
number of nominees. Unfortunately, the expected variances of the 
distributions of these different groups were highly dependent on the 
number of nominators in the groups and, therefore, an adjustment in 
peer nomination scores was required to equate the expected variances. 

To deal with the problem, a formula was developed to adjust nomination 
scores, resulting in the expected variances being equated for flights^ 
with differing numbers of nominators. Then, these adjusted nomination 
scores (for each grouping of flights separately) were aggregated in- a 
single frequency distribution, and a percentile score was assigned to 
each score. Finally, standard scores were developed for each adjusted 
nomination score based on the corresponding percentile of a normal 
distribution. 

In general, these adjustments in raw scores allowed statistically 
justified comparisons to be made of peer nomination scores from 
different flights. Importantly, the distributions of adjusted scores 
for individual flights eliminated characteristics of the raw score 
distributions known to be artifacts of the peer nomination measurement 
process. On the other hand, these "new" distributions did retain the 
characteristics that reflect valid variations across the different 
groups of nominees. For example, with this scoring system if one 
flight had two or three trainees consistently nominated positively on 
a rating category, and a second flight in the same grouping had 
positive nominations on this category spread more evenly across flight 



^Many flights contained more nominees than nominators. 
However, for some flights there were many more nominees than 
nominators and in other flights almost every nominee served as a 
nominator. Thus, when flights were grouped according to number of 
nominees, the number of nominators varied, considerably in some cases. 
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members, then those two or three in the first flight would tend to 
receive higher standard scores than the top two or three in the second 
flight. In sum, the scoring procedures outlined above appeared to 
meet the critical requirement of allowing justified comparisons 
between trainees across flights, and therefore standardized scores 
were computed for each trainee on each rating category and these 
scores were used in correlational analyses. 

Method for Evaluating the Interrater Reliability of the Nominations 

The approach to obtaining interrater reliabilities involved computing 
special intraclass correlation coefficients for each rating category 
within flights and then examining the distributions of those 
reliabilities for each category. Derivation of a more general 
formulation for the intraclass correlation was required because no 
existing formula was appropriate for this data set. In particular, 
the data suffered from the "diagonal problem," i.e., trainees in the 
sample were instructed not to nominate themselves, and this created 
undefined observations in an otherwise completely crossed experimental 
design. Gordon (1969) and Willingham (1959b) have provided methods 
for estimating reliabilities under these conditions but, unfortu- 
nately, neither was totally appropriate for this data set because some 
nominees did not serve as nominators, as has been previously mentioned. 

Accordingly, a more general intraclass correlation was developed to 
meet the requirements of this data set. It is of the general type 
presented in Hoyt (1941). 

rkk ^§ b. 

MSB 



where: 



r|^,^ = the intraclass correlation corresponding to k raters 
MSb = the mean square between nominees' uncorrelated mean 

scores^* 
MSe = the mean square error 



The formula derived was then used within each flight to compute the 
interrater reliability for each of the eight rating categories. The 
resulting reliability estimators, r^k values, were not directly 
comparable across flights with differing numbers of nominators because 



^Uncorrected scores could be used at this point because all 
analyst were carried out within flight, thereby avoiding the problem 
of non-equivalent scores across flights. 



the reliabilility to be accepted increases with the number of 
nominators (i.e., flights with 50 nominators would be expected to 
provide more reliable ratings than flights with 25 nominators, just as 
a test of 50 items is expected to be more reliable than one with 25 
Items). Thus, to provide estimators which could be considered 
comparable, the Spearman-Brown prophecy formula was applied to adjust 
each estimator to a common number of nominators as followis: 



^37. 37 -7-37 



where: 



k = harmonic mean of the number of nominators nominating each 
nominee in the flight 

This coefficient reflects the level of interrater reliability adjusted 
to 37 nominators which is approximately the mean number of nominators 
per flight. This adjustment permits fair comparisons of reliabilities 
across flights, and the distributions of these interrater agreement 
indices were examined for each rating category separately. 

In order to describe the norming and standardizing of nomination 
scores and the derivation of a general interrater reliability formula 
used in the research, first configuration of data and some terms must 
be defined. 



Description of Data 

Consider Figure 1, the layout of data for one of eight dimensions in 
any one of the 596 flights of trainees. Recall that nominators 
attending a rating session were allowed to nominate anyone in their 
flight (except for themselves), including those not present at the 
session. Therefore, rating data from each flight involved M nomin- 
ators and N nominees, where N >^M. 

Again referring to Figure 1, for each of the NM combinations of 
nominators and nominees, a value, Vi,. (i = 1, 2, . . . N; j = 1, 
2, . . . M), was assigned, except where the nominee and nominator 
coincided--this case is undefinable since nominators were instructed 
not to nominate themselves. Thus, there were M(N-l) values assigned 
according to one of three scoring rules: 
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1. Positive nominations only: 

Vii = 1 nominee was one of the five positive 

nominations made by the jth nominator; 

Vii " undef inable; 

Vii ^ ^ nominee was not one of the five 

positive nominations made by the jth nominator. 

2. Negative nominations only: 

Vii = -1 if the i**^ nominee was one of the five negative 
nominations for the j^^ nominator; 

Vii " indefinable; 

Vii = 0' if the i**^ nominee was not one of the five 
negative nominations for the j^h nominator. 

3. Positive plus negative (algebraic sum) nominations: 
vij = 1 if positively nominated; 

Vij = -1 if negatively nominated; 

Vii " undefinable; 

Vij = 0 if not nominated. 

Initially, the nominations were scored all three ways for examination 
of reliability. 

Raw scores, Xi, were then computed as the arithmetic mean of 
available scores, Vij, for the i*"^ nominee, i.e., 

Xi = 2 Vij Z (1) 

j=l 
(i/j) 

where* 

Z = M-1 for all i 1 M and 
Z = M for all M < i < N. 



H 
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vij is the value assigned to the 1*'^ nominee by the j*h 
^ nominator (according to one of three sets of scoring rules) ^ 

* indicates that values on the diagonal are undefinable because 
nominators could not nominate themselves. 



Figure 1. Layout of peer nomination data for one flight of trainees 
on one of eight dimensions. 



is the raw score (mean) for the ith nominee: 
M 

j=l Vij/Z 
(i^j) ' 

where 

Z = M-1 if i iM, and 

Z = M if M < i ^ N. 

is the mean of nominations for the j^l^ nominator 
(ordinarily a constant if data are complete): 

Mi = 2 Vij /N-l 

m) 



X is the grand mean of scores: 
N 

i=l 



X. = 2 Xi/N 
•1=1 / 



Figure 1 (continued). Layout of peer nomination data for one flight 
of trainees on one of eight dimensions. 
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For the case in which positive nominations only are scored, the range 
of Xi is therefore 0 to 1; for negative nominations only, the range 
is -1 to 0; and for the algebraic sum nomination scoring, the range is 
-1 to 1. Reliability of these nomination scores for each of the three 
scoring rules was computed within flights. The algebraic sum scores 
proved to be most reliable and all subsequent adjusting of scores was 
accomplished for these scores only (see Table 1). 



Table 1. Interrater Reliabilities of Positive, Negative, and 
Algebraic Sum Nominations^ 



Scoring Methods 
Positive Negative Algebraic Sum 



1. 


Try Hard 


87b 


93 


93 


2. 


Cooperative 


83 


90 


93 


3. 


People Oriented 


84 


87 


89 


4. 


Calm/Emotionally Stable 


80 


88 


88 


5. 


Bright/Intelligent 


87 


91 


93 


6. 


Not Excitable 


79 


92 


91 


7. 


Physical Strength/Energy 


92 


90 


93 


8. 


Likely to Succeed in Air Force 


90 


93 


94 



3Mean reliabilities were computed separately for the male 

(N = 432) and female (N = 164) flights, and these means were very 

similar. Therefore, ony the reliabilities of the combined groups are 
presented here. 



^Decimals have been omitted. 



Developing Standard Scores for Nominees 

The strategy for creating nominee scores that may be legitimately 
compared across flights included an empirical norming step along with 
the equating of expected variances for different flights. 
Accordingly, groupings of flights were formed with approximately the 
same number of nominees in each flight. Within each of these group- 
ings, then, the level of kurtosis of the distributions of raw 
nomination scores is very similar, and this helps to create comparable 
raw scores across flights within these groupings. However, because 
the number of nominators for flights within each of the groupings 
varied somewhat more than the number of nominees, and differential 
numbers of nominators also affect the comparability of scores across 
flights, these scores were adjusted to equate the expected variances 
in nomination scores for the different flights (again within group- 
ing), which in turn aids in creating scores that are comparable across 
the flights. 



n 



At this point, the adjusted nomination scores for each grouping were 
pooled forming a single frequency distribution, and percentile scores 
were assigned. Finally, standard scores were assigned to each 
percentile score based on the corresponding percentiles of a normal 
distribution. Details of these procedures follow. 

Two assumptions were required to proceed with the adjustmtffll Of 
nomination scores: (a) the underlying distribution of "true" 
nomination scores in large groupings of trainees is normal, and 
deviations from normality are artifacts of measurement; and (b) the 
reliability of nominations from any one nominator selected at random 
is the same as the reliability of any other nominator. The first 
assumption is necessary to justify forcing a normal distribution of 
scores for each grouping of trainees (with similar numbers of 
nominees) during the final step of the empirical norming process. The 
second asumption is required to make reasonable the adjustments of 
nomination scores using a formula that equates expected variances in 
the distributions of those scores across flights. 

Thus, flights of trainees were aggregrated into 10 groupings, with 
each grouping having a range of three or fewer nominees across the 
flights (e.g., 30-33 nominees). Then, for any one of the rating 
categories, the following summary statistics of the raw scores x were 
computed; 

Mx = mean of all raw scores on a given rating category within 
a grouping of flights 

= variance of raw scores 



k = mean of the averages (harmonic means) of nominators for all 
all of the flights within the grouping. 

Using these statistics, the variance of the "typical" nominator's 
scores could be approximated by employing a formula given by Gulliksen 
(1951, pp. 73): 



where the reliability value, r^k is the reliability of the raw 
nomination scores of the "typical flight" within the grouping , i.e.. 



X 




(2) 



k 

37" 



r 



37, 37 



(3) 



kk 
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This is the level of reliability adjusted to the average number of 
nominators in the flights within the grouping. The variance of a 
"typical" nominator's scores, given in equation (2) above, is 

(11) 

important for adjusting nomination raw scores. 

Now, again within each grouping of flights, adjusted raw scores, x*, 
were computed so as to have a mean of zero and an expected variance of 
approximately (25)2 as follows: 

= g(x -X.) (4) 

where x. = mean nomination within flight (see Figure 1) and 




X is in turn the theoretical standard deviation of the nomination 
scores for flights within the same number of nominators, k, within the 
grouping of flights, i.e.. 



2 . 



= a 



(11) 



k + k (k - 1) rii 



(5) 



Equation (5) also comes from Gulliksen (1950). The result of this 
particular adjustment to each raw score in a grouping of flights is a 
distribution of adjusted raw scores, x', with a range of -100 to +100 
and a standard deviation of 25. The adjusted scores were rounded to 
the nearest integer and percentile scores obtained for the frequency 
distribution of these rounded scores. Then, finally, standard scores 
were assigned to each percentile score based on the percentile 
equivalents of a normal distribution. For example, a T-score of 60 
would be assigned to the 84th percentile, where the 84th percentile is 
one standard deviation above the mean in a normal distribution. 



In this manner, nomination standard scores were developed which are 
comparable across flights. This comparability rendered possible 
correlational analyses with the nominations, incorporating data from 
trainees in all flights. 

Reliability Estimation of Individual Flights 

Computation of interrater reliability for peer nominations poses 
special difficulties because of the "diagonal problem"--i.e. , 
nominators cannot nominate themselves. This creates an undefined 
observation in the otherwise completely crossed design. 

Two authors have previously offered formulas for computing reliability 
estimates under these conditions (Gordon, 1969; Willingham, 1959b) . 
However, neither of these formulas was suitable for the data collected 
in this study because neither was readily adaptable to the case where 
some nominees did not serve as nominators. 
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Thus, it was necessary to develop an appropriate index of interrater 
reliability for the purposes of this study as follows: 

MS - MS 

r = _B E (1) 

kk MSb 



which is the Hoyt (1941) Reliability formula for k number of 
nominators given the following definitions (see Figure 1): 

k 2 (xi - x.)2 

MSB = Jl! (2) 

N - 1 



and 



M 
2 



MSe = 



2 j=l /Vij - Mi - Xi + x.\2 

i=l \ \ .. .. ) (3) 

and k is the harmonic mean of the numbers of observations used in 
computations of the X^: 



/m N \ 

\i-i"-^ i=M.iy 



k =/ s _!_ + 5 -^1 \ 

(4) 



M + N-M 
M-l M 

The rationale for the estimator, r^i^, follows from conventional 
reliability theory. The parameter of estimations is: 

Pkk= <5) 

where o2 is the variance of "true" scores, k is the theoretical 
number of nominators used to obtain the vector of observed scores, 

and is the variance of errors. 
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First, MSe is defined in Equation (3) to be an estimator of a^, i.e., 

e 

est (a§) = MSe (6) 

which loses M degrees of freedom because of the undefined values, 
vii. Under the usual assumption that there is no statistical 
interaction of nominators with nominees, in an analysis of variance 
sense, the estimator, MSe, be shown to be unbiased. Failure of 
this assumption to be realized would lead to an overestimate of 
and, consequently, an underestimate of P|^|^. 

The estimate of observed variance in scores, MSg, may be shown to 
consist of two parametric components 

E(MSb) = ^2 + kq2 

If the asumption of independence of errors is made. Specifically, it 
is assumed that 

Xi = Ti + ii (8) 

where is an observed score for nominee i and is the "true" 
score value for this nominee while e^ is the independently 
distributed error component. This means that is independently 
distributed with an expected value of and variance of 

a|.=a2/z (9) 

where Z = M-l if i<. M and Z = M if i > M. The reader may recognize 
Equation (9) as the v?iriance of the mean error across the nominators 
for the i^^ nominee by the central limit theorem. 

Finally, it can be seen that the numerator of Equation (1) has an 
expectation equal to the numerator of Equation (5) and, also, the 
denominator of Equation (1) has an expectation which is equal to the 
denominator of equation (5), i.e., 

E(MSb - MSe) = (10) 

and 

E(MSb) = + ^1 (11) 
which are the conventional conditions of reliability estimation. 
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Thus, Equation (1), with its elements defined in Equations (2) and 
(3), provided the general intraclass estimator for assessing the 
reliability of nominations in individual flights. Next, reliability 
information was summarized to depict the stability of the nominations 
across the many flights. 



Reliability Estimation for All Flights 

Reliabilities werie computed within each flight and for each rating 
category using Equation (1). Then, each of these reliabilities was 
adjusted to provide estimates that (a) yield fair comparisons of the 
degree of reliability for different sized flights, and (b) give a 
realistic picture of the reliability levels to be expected in future 
peer evaluation research with Air Force basic trainees. 

The rationale for the adjustments is that the magnitude of reliability 
depends partially on the number of nominators providing ratings. The 
larger the number of nominators, the greater the expected level of 
interrater reliability, just as the longer form of a test typically 
provides higher reliabilities than a shorter form. Importantly, 
increments in reliability in both of these cases can be closely 
estimated by the Spearman-Brown formula, and we used this principle to 
"correct" reliabilities (up or down) to the 37 nominator level using 
the formula: 



The 37-noininator level was used because it is approximately the mean 
number of nominators in the 596 flights studied, and it is assumed 
that future peer evaluation research with basic trainees will involve 
groups of approximately the same size. Therefore, these 37-nominator 
reliabilities should provide good estimates of the level of 
reliability that mght be expected in future research. 




(12) 



where k = number of nominators in the flight 



r|(k - reliability of the nominations in this flight 

^37 37 = the reliability adjusted to the 37-nominator level. 
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III. RESULTS 



Interrater Reliability 

Reliabilities for the positive, negative, and algebraic sum* of the 
positive and negative nominations were computed for each flight 
separately. To aid in selecting a scoring paradigm for the 
nominations, the means of these reliabilities were then compared. 
Rationale for using mean reliabilities was as follows: First, the use 
of means was motivated by the practical need to characterize the 
reliabilit-'es with single summary indices. And second, it was 
anticipated that differences between this kind of estimate of "average 
reliability" and other estimates that might be developed would be very 
slight because of the "tight" distributions experienced (see Table 2 
for a depiction of these distributions). 

Though it may be possible to develop a single estimator within an 
analysis of variance framework, it was considered impractical because 
of the unnecessarily complex statistical development which would be 
required to weight appropriately the variance components of the 
individual flights containing differing numbers of nominators. 
Moreover, the computer programming for such an estimation process 
would have been considerable. Accordingly, the mean of the intraclass 
estimators for a given dimension (which adjusted using the 
Spearmen-Br own pr oph ecy f ormu 1 a to a common f r ame of reference — 37 
nominators) was accepted as a reasonable estimate of the reliability 
of a randomly selected flight with 37 nominators. 

Table 1 clearly shows that the algebraic sum scores are most reliable, 
although differences between these reliabilities and those of the 
negative nomination scores are small. This slight superiority in the 
stability of the algebraic sum scores suggests an advantage of using 
these scores in subsequent research. In addition, the summed scores 
contain more information than either the positive or negative scores 
alone, and this feature of the summed scores also argues for their use 
in future research. 

Table 2 provides a more complete picture of the reliability results 
for the summed scores. It showed that . reliabilities within flights 
are uniformly high, and therefore, that very stable peer nomination 
data can be expected for flights of this size (i.e., Ns « 37). 



^aw Scores for the algebraic sum scoring procedure were 
computed by simply adding together all positive nominations and 
subtracting the sum of the negative nominations. 
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Development of Norm Tables for Scoring Nominations 



Norm tables yere developed to score these nominations or nominations 
gathered in future peer evaluation research. The tables permit the 
researcher to compute a standard score for any raw nomination score. 
The tables have been prepared for many different sized flights, and 
they are presented in Appendixes A to H. 



IV. SUMMARY AND CONCLUSIONS 

The Air Force gathered peer nominations on over 27,000 basic 
trainees. Interrater reliabilities of the positive, negative, and 
algebraic sum nominations were computed within flight, and the 
distributions of these reliabilities were examined. Reliability of 
the algebraic sum nominations proved to be highest; means of the 
intraclass correlations used to index reliability ranged from .88 to 
.94 for the eight rating categories, indicating good stability for the 
summed nominations. 

A special scoring strategy for the summed nominations was then 
developed to render comparable raw nomination scores across the 
different flights. The standardized nomination scores on each of the 
eight rating categories were subsequently correlated with test scores 
available for many of the trainees. The patterns of these 
correlations provided some evidence of convergent and discriminant 
validity for the nominations. 

Conclusions based on this research are: 

1. In future peer evaluation studies, the Air Force should continue 
to gather both positive and negative nominations. The algebraic 
sum of these nominations appears to offer the most reliable 
measure of the rating categories. 

2. The strategy developed for standardized scoring of nominations 
should be used in future peer evaluation research. The norm 
table developed in this research will greatly facilitate the 
adjusting of raw nomination scores. The capability of correcting 
nomination scores to achieve comparability in such scores across 
flights, along with the high reliabilities of these nominations, 
further substantiates the utility of collecting peer ratings. 
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APPENDIX A. NORM TABLES FOR DERIVING STANDARDIZED 
PEER NOMINATION SCORES AND NORM TABLES FOR 
DIMENSION A - TRY HARD 



These appendices A to H provide instructions and tables for scoring 
peer nominations on each of the eight rating categories used in this 
research . The scori ng procedures for the f i rs t category. Try Hard , 
are described in Appendix A; the same procedures can then be used to 
score nominations on the other rating categories. 

First, the rating instructions require trainees in each flight to 
identify the five best or most effective trainees in their flight on 
each rating category. They are also instructed to identify the five 
worst or least effective trainees in their flight on each of these 
categories. Now, a given trainee's raw score (X) for a category is 
computed using the formula: 

j( g Number of positive nominations - Number of negative nominations 

Number of nominations possible 

Because trainees are prohibited from nominating themselves, the number 
of nominations possible is one less than the total number of 
nominators in the flight, unless the trainee whose nominations are 
being scored does not serve as a nominator (e.g., he/she is absent 
from the rating session). In that case, the number of nominations 
possible is equal to the number of nominators in the flight. 

After the raw score for a nominee has been computed for the rating 
category, an adjustment must be made in this score to corrrect for the 
effects on the raw scores of differing numbers of nominators and 
nominees. This can be accomplished by entering Table A-1 with the 
number of nominators and nominees in the trainees* flight and then 
employing the obtained value (Q) in the equation 

Y = Q X 

where Y is the transformed raw score* 



*In General, these scores will range from -100 to +100. It is 
mathematically possible for them to take on values greater than +100 
or less than -100 but this should occur infrequently. Also, we should 
mention that the transformation indices in Tables A-1, B-1, etc., are 
in the main based on large norming groups and therefore are probably 
stable estimates that would be confirmed in future peer rating 
research with these rating categories and the same kind of basic 
trainee sample. However, as the Ns in these tables indicate, the 
values for the smallest group sizes (e.g., N = 29) and the values for 
the largest group sizes (e.g., N = 58) are based on smaller samples, 
and therefore, the adjustments to raw scores made for these groups 
should be performed with caution. 
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The standardized score for the trainees on this rating category can 
now be obtained by entering Table A-2 with the transformed raw score 
and the number of nominees in the trainee's flight. An example is now 
provided to show the scoring procedure: 

Number of nominators = 37 (the trainee did provide nominations) 
Number of nominees = 40 

Number of positive nominations for the trainee on Category A = 14 
Y _ 14-4 

^ " ~36~ 
X = + .278 

The Q value from Table A-1 is 92.534 
Thus, Y = +.278 (92.534) 
Y = +25.72 or +26 
and the standardized score (from Table A-2) = 62 

The same procedures should be followed in scoring nominations on other 
rating categories. The requisite tables are included in Appendixes B 
to H. 

As mentioned in the text of the report, this standardizing process 
yields scores that are comparable across flights with differing 
numbers of nominators and nominees. Accordingly, the standardized 
scores permit analyses of data from multiple flights, a critical 
requirement for conducting further research that focuses on 
correlations between peer nominations and criterion variables. 
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^0 


(»0 


40 


39 


39 


39 


39 


-23 






dO 


dO 


«f0 


39 


39 


39 


39 


-22 






l>0 


kl 


IfO 




dO 


39 


39 


-21 








kl 


(fO 




dO 


i»0 


k^ 






m ^ m mm 






mm mmm 






mmmmm 





58-;) 
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TABLE A*2. TABLE OF STANdARO Si 
FOR DtHENSZBN A • 



TRANSFORMED 






N-J-N-B-e-R 


RAM-SCORE 


29-29 


31-33 


3(>-36 


37-39 








mm m 


«a mmm 








ki 


ki 


-19 


%1 




ki 


kZ 


-le 


«l 


(i2 


ki 


kZ 


-ir 




^2 


ki 


kZ 


-16 




^2 


kZ 


kZ 


•IS 




L9 
HC 




kZ 


-u 


^2 


%3 


kZ 


kZ 


•13 


^3 


%3 


kS 


kZ 


-12 


(i<» 


kZ 


kZ 


kk 


-11 


kk 


kk 


kk 


kk 


-10 




Hp 


kk 


kk 


-9 




<»5 


kk 


kk 


-8 


<»6 


kb 


k$ 


kS 


-f 




kf> 


k5 


kS 


-6 




k7 


ki 


kf> 


•5 


k7 
11 


LI 


kh 


kh 


-i» 


<»7 


k7 




<>6 


-3 


LA 




k7 


k7 


-2 


•►9 


<»8 


k7 


ki 


-1 


<»9 




ki 


<»9 


0 


50 


50 


ki 


i»9 


1 

• 




CI 


V9 


50 


2 


51 


51 


V9 


50 


3 


52 


52 


50 


51 


% 


53 


53 


51 


52 


S 


53 


53 


51 


52 


D 






52 


53 


? 


5i» 


5i» 


53 


5k 


B 


5i» 


Sk 


53 


5k 


9 


55 


Sk 


Sk 


55 


10 


55 


5k 


ok 


55 


11 






35 


56 


12 


56 


56 


55 


56 


13 


56 


56 


56 


57 


U 


56 


57 


56 


57 


15 


37 


57 


57 


58 


16 


57 


57 


57 


58 


17 


57 


57 


57 


58 


18 


56 


58 


56 


59 


19 


58 


56 


58 


59 


20 


58 


58 


59 


59 



ORES FOR TRANSFORHEO RAM-SCORES 
TRY HARD* 

PAGE 3 OF 5* 



O-F 


M-O-H-I- 


'N-E-E-S 








<«0-%2 


^3«i»5 


i»6-<>8 


49-51 








• ••• • 










ki 


<»0 




ki 




40 


ki 


ki 


i»0 


ki 




■fU 


ki 


ki 


i»l 


40 


(lO 


ki 


ki 


ki 


i»l 


41 




kl 

HI 


kZ 


ki 


ki 


41 


ki 


til 
HI 


kc 


kZ 


kZ 


41 


ki 




kZ 


kZ 


kZ 


42 




HI 


kZ 


kZ 


kZ 


42 


ki 


kl 
Ht 


kZ 


kZ 


43 




"ft 


Ht 


kZ 


kZ 


kZ 


43 


"ft 


Ht 


kk 


kZ 


kZ 


43 


42 




kk 


kk 


kZ 


43 


kZ 


H** 


kk 


<»<» 


kk 


44 


kl 
■f W 


kS 
H* 


k5 


k5 


kk 


44 


kk 


kk 
"ft 


k5 


^5 


k5 


44 


kk 
^■f 


kk 
HH 


h6 


V5 


k5 


45 


(»5 


kk 


<»6 


i»6 


i»6 


46 


<»6 


<»5 


'»7 


<»6 


i»6 


46 


^6 


kS 


k7 


k7 


<»7 


47 




kX 
HO 


ki 


ki 


ki 


46 


"f 0 


LI 
Hf 


<»9 


<i9 


>»9 


~7 




HO 


50 


50 


50 


50 


50 


<»9 


50 


SO 


50 


51 






51 


51 


51 


52 


52 


pi 


51 


52 


52 


52 




9t 


52 


52 


52 


53 






53 


53 


53 


53 


5<i 


53 


53 


5k 


5k 


54 


5(a 




5k 


5ii 


5k 


55 




7H 


55 


53 


55 


55 


5& 




55 


55 


55 


56 




55 

77 


56 


56 


56 


56 


57 


55 


56 


56 


56 


57 


57 




57 


57 


57 


57 


5A 


57 

7r 


57 


57 


57 


58 


58 


57 

7 1 


58 


56 


58 


58 


5A 


5A 

7Q 


58 


56 


58 


59 


59 


58 


58 


59 


59 


59 


59 


59 


59 


59 


59 


60 


60 


59 


59 


59 


60 


60 


60 


59 


60 


60 


60 


60 


60 


60 



ERIC 



0 



TABLE A-2. TABL£ OF STANSARO SCORES FOR TRIN5F0RHED RAH-SCORES 
FOR OtH-NSION A • TRY HAROi 

PACE k OF 5. 



TRANSFORMED 










O-F 


_ii_T - 










RAH-SCORE 


29-29 


91*99 


5i»-36 


97-99 


f n f.9 






h9-5a 


C 9.CI. 






••••• 


mmmmm 














• • 49 W 




21 


56 


99 


39 


60 


60 


60 


60 


61 


61 


60 


22 


56 


59 


60 


60 


61 


61 


61 


61 


61 


60 


Z3 


59 


60 


60 


C. A 

60 


61 


c« 
61 


CI 

61 


cl 

61 


Cl 

61 


Cl 

61 




59 


6v 


61 


dI 


61 


el 
OA 




c9 
DC 


DC 


cl 

OA 


zs 


59 


60 


£.4 

61 


61 


6^ 


c9 


C9 
OC 


c9 


C9 

be 


Cl 

bi 




59 


60 


62 


61 


62 


62 


62 


63 


62 


62 


27 


60 


61 


62 


62 


62 


63 


63 


63 


63 


62 


Z8 


6P 


61 


59 


e 9 

6Z 


cT 

69 


ci 
69 


C7 
D« 


CT 

bo 


C? 

bo 


c? 
oc 


c9 


61 


61 


ok 


6c 


69 


cv 
69 


CV 

6« 


Cf. 

bH 


c? 
b* 


'1 

90 


til 


61 


61 


bk 


6£ 


69 


ci 


CL 
OH 


CL 


CL 

b4 


Cl 
00 


3t 


63 


61 


6i» 


63 




6V 


6<» 




6i» 


63 


32 




62 


65 


63 


6^ 


bk 


6^ 




6I» 


bk 


93 


6^ 


6Z 


97 


6<^ 


hk 


£.9. 

6^ 


65 


ce 
65 


cc 
65 


09 


9* 




bZ 


69 


6h 


cc 
65 


cc 
oo 


ce 

00 


ce 
bo 


cc 

07 


CC 

00 


95 


65 


69 


66 


6h 


65 


69 


ce 
65 


ce 
65 


CC 

65 


C£ 

DO 


36 


65 


63 


63 


6<» 


65 


65 


65 


66 


65 


66 


11 


65 




66 


65 


66 


66 


66 


66 


66 


66 


98 


65 


65 


Of 


65 


CC 

66 


CC 

66 


cc 
00 


cc 
bo 


cc 
bb 


C7 

bf 


la 


65 


6d 


Of 


65 


cc 

00 


c& 

09 


I^C 
00 


cc 
00 


CT 
Or 


Df 


1 A 


65 




Of 


bo 


cc 
00 


C? 
Of 


CT 

Of 


c? 
Of 


CT 
Of 


Df 




66 


66 


68 


66 


67 


67 


57 


67 


67 


68 




66 


67 


S9 


67 


67 


67 


67 


57 


67 


68 


«>9 


66 


67 


o9 


C 9 

6f 


c ^ 
Of 


1 9 

Of 


t9 
Of 


C9 

bf 


CA 

bfl 


CA 

00 




67 


68 


09 


6f 


DO 


CI 

oO 


CA 
00 


CA 

bO 


CA 

bO 


CO 

bo 




69 


oB 


69 


68 


C M 

68 


C A 

DO 


CA 
OO 


C A 

b6 


CA 

bO 


CO 

Do 


(»6 


70 


69 


70 


68 


66 


68 


68 


68 


69 


69 


^7 


71 


70 


70 


68 


69 


69 


68 


69 


69 


70 


K^B 




71 


71 


68 


69 


59 


69 


69 


ca 

69 


7A 

7*1 


^9 




71 


71 


69 


9 A 

70 


ca 

69 


69 


69 


CO 

69 


7U 


$0 


■ ■ 


7Z 


71 


69 


9A 

70 


9A 

f 0 


59 


69 


CO 

69 


9| 

71 




mm 


72 


72 


70 


70 


70 


70 


70 


69 


71 


52 


m m 


72 


72 


70 


71 


70 


70 


70 


70 


71 


5J 


mm 


7c 


7Z 


71 


71 


9| 

71 


9 A 

70 


9A 

70 


9A 

7U 


99 

7C 


5^ 


mm 


73 




71 


7X 


91 

71 


9l 

71 


91 

71 


9A 

70 


99 

7c 


55 


mm 


79 


75 


« 4 

71 


71 


91 

f I 


9| 

71 


9| 

71 


9n 

70 


7> 

ft 


56 


mm 




7C 
f 3 


v9 

ft 


77 
ft 


91 

f 1 


yi 

f A 


f 1 


7fl 


f w 


57 


mm 


73 


76 


72 


72 


72 


71 


71 


« A 

70 


73 


56 


mm 


7i» 


75 


72 


73 


72 


72 


72 


71 


73 


59 


m m 


7(i 


75 


72 


73 


73 


72 


72 


71 


7l» 


60 






77 


72 


7k 


7J 


72 


72 


71 


7^ 


• m mmmm • * a aa 


mm^mm 
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TA8LE 1-2. TABLE OF STANOARO S 
FOR DIHENSZON A • 



TRANSFORHEO 








RAW*SCORE 




31-33 


•'If 








• •••• 






61 


mm 


mm 


77 


73 


sz 


mm 


mm 


77 


73 




mm 


mm 


78 


73 


6% 


•m 


mm 


78 


71* 


69 


mm 


mm 


78 




$6 


mm 


mm 


78 


75 


67 


mm 


mm 


79 


75 


66 


mm 


mm 


79 


75 


69 


mm 


mm 


79 


76 


70 


mm 


mm 


80 


f Q 


71 


mm 


mm 


80 


77 


72 








77 


73 








77 


7* 








7ft 


mm 

75 








f 0 


76 








80 


77 








83 


76 










79 










60 


mm 


mm 




• m 


61 











62 
63 
6% 
65 

66 
67 
66 
69 
90 



ORES FOR TRANSFO^HEO RAM-SCORES 
TRy HARD. 

PACE 5 OF 5. 



0-F N-O-H-I-M-c-E-S 













If a * A 

58-78 










• •••• 




7k 


73 


73 


72 


71 




7k 


73 


73 


72 


71 


mm 


75 


7k 


73 


73 


71 


mm 


75 


7k 


74 


73 


72 


mm 


76 


1 7 




f * 


7? 
ft 


m m 


76 


75 


75 


73 


72 


mm 


77 


76 


75 


73 


73 


mm 


77 


7S 


76 


73 


7k 


mm 


78 


7S 


76 


7<» 


7k 


mm 


78 


77 


r D 


7L 
f H 


75 




78 


77 


77 


7k 


76 


mm 


79 


. 78 


77 


7k 


76 


mm 


79 


76 


78 


75 


76 


mm 


79 


78 


78 


75 


78 


mm 






fo 


7b 


m A 

78 


mm 


81 


79 


79 


76 


78 


mm 


81 


79 


79 


76 


79 


mm 


82 


79 


79 


77 


79 


mm 


82 


80 


80 


77 


79 


mm 






ill 


78 


79 


mm 


83 


81 


81 


79 


79 


mm 


83 


82 


81 


80 


79 


mm 


ik 


82 


82 


80 


80 


mm 


tk 


82 


8i» 


81 


80 


mm 


tk 


8^ 


89 


82 


80 


mm 


85 


85 


mm 


83 


80 


mm 


85 


85 


mm 


83 


80 


mm 


85 


88 


mm 


8I> 


81 


mm 


86 




mm 


85 


81 


mm 


86 




mm 


86 


81 


mm 



TABLE fl-l. TABLE OF IMITIlt, WH-SCORE TRAMSFORHITIOM COEFFICIEMTS 
FOR DIMENSION B • COOPERATIVE. 



N0« Or 






M-U-H-B*E-R 


NONINftTORS 






dH*Ob 




•••••••••• 


••••••• 








21 


85.009 


67.655 


95.367 


92.595 


22 


85.3<>0 


87.996 


AC 9l9 

95«737 


o7 act 
9Cft990 


29 




ftft tin 

o6«3lU 


9D«Uf 0 


a3 f ftk 


ck 


S7«9Z9 


Aft ftOI 


A£ 10l> 


7* •77* 


Z5 




ftft ft79 


0£. Cftft 


03 ft7G 


Z6 


S6tH37 


ftA 4 9l 


A^ AC 4 


Alt 1 Ikfl 


Z7 




ftA YCO 




3ft7 


Z8 


A £ ft i II 


fta evft 


A 9 I^CC 


i%L ftl ft 


29 


• V ft A A 


ftA 9ft C 


AT &7ft 

9r •br 9 


QL ftSC 
9*l«0»7 


JO 




ftft Il7ft 






31 




90.U1 - 


9Si087 


95.230 


32 




90.333 


Oft 273 


77«T** 


n 
00 




7* • H9> 


Aft LUO 


77«7Q* 


3* 






aft ftlft 


77«f 


35 






Qft^77L 


fle^ftac 

77ftO»Q 


36 






Qft.Q^L 
7Dt7£H 


Ofk^fl&2 
7w*''f 


97 








Oft.iUDl 
7D«*0*' 


36 








06^211 


39 






mmm 


7w*9*w 


fill 










^1 






mmm 




i»2 


— 




mmm 














i.f. 










«i5 












• 






mmm 


H7 








mmm 










mmm 


w 








mmm 


5« 






mmm 


mmm 


SI 






mmm 


mmm 


5t 






mmm 


mmm* 


99 






mmm 


mmm 


5k 






mmm 




59 




mmm 


mmm 


"•1 


56 




mmm 


mmm 




57 




mmm 


mmm 




58 




mmm 


mmm 


mmm 








••••••• 




NORMATIVE 








1107 


SAMPLE N> 


29 


1Z9 


390 



M-O-N-I-N-E-E-S 





%3«^5 


i»6-^8 


49-51 


52-34 


58-58 




••••••• 








••••••• 


100.213 


105.232 


i Mm a ft 4 

109*101 


114*650 


lcD*rD4 


lie LQL 


100.600 


105.638 


109*521 


115*091 


121*249 


116*943 


100.958 


106.012 


109*909 


115*498 


121*677 


117*355 


101.289 


106.359 


110*268 


115*875 


122*074 


117*737 


101.596 


106*681 


110*602 


116*225 


122*442 


118*092 


101*882 


106.981 


110*912 


116*551 


122*785 


116*422 


102.U9 


107.261 


111*202 


116*656 


123*105 


118*731 


102.399 


107.523 


111*^73 


117*140 


123*405 


119*019 


102.633 


107.769 


111*728 


117*407 


123*686 


119*290 


102.853 


107.999 


111*966 


117*658 


123.950 


119*544 


103.060 


108.217 


112*191 


117*894 


i9L 10ft 


110. 7A3 


103.256 


108.^21 


112*<>03 


116*116 


124*432 


120.809 


10 3. HO 


108.615 


112*604 


116*327 


124*654 


120*222 


103.615 


108.798 


112*793 


116*526 


124*863 


120.423 


103'.780 


108*971 


112*973 


116*714 


125*061 


120*614 


103*937 


109*136 


113,1^ 


118*893 


125*249 


120*795 




109*292 


113*305 


119*062 


125.428 


120*967 


10^.228 


109* ^^1 


113*459 


119*224 


125*598 


121.131 


lOb.363 


1II9«582 


.113*605 


119*378 


125*760 


121.287 


I0k«492 
ft 


199«717 


113*745 


119*525 


125*914 


121.436 


10^.615 


109*8^6 


113*879 


119*665 


126*062 


121.578 


10k. 732 

**9f 1 


109*963 


1U*006 


119*799 


126*203 


121*713 




lis* 087 


114*128 


119*927 


126*337 


121.843 




110*200 


114*245 


120*049 


126*466 


121.%7 




110*308 


114*357 


120*167 


126*590 


122.086 






114*464 


120*280 


126*709 


122.201 






114*567 


120*388 


126«823 


122*310 


mmm 




114.666 


120*492 


126*932 


122.416 


mmm 


mmm 




120*592 


127*037 


122.517 


mmm 


mmm 




120*688 


127*138 


122,614 


mmm 


mmm 


mmm 


120*781 


127*236 


122.708 ^ 


mmm 


mmm 


mmm 




127.330 


122.799 


mmm 


mmm 


mmm 


mmm 


127*421 


122*886 


mmm 


mmm 


mmm 


mmm 


127*508 


122*970 


mmm 




mmm 


mmm 




123*052 


mmm 




mmm 


mmm 




123*130 


mmm 




mmm 


mmm 




123*206 


■ mmm 


mmm 


mmm 


mmm 




123*280 








••••••• 




••9M«* 


2924 


6807 


8968 


5525 


1216 


58 



ERIC 



TABLE B-2. TA8LE OF STANDARD SCORES FOR TRANSFORNEO RAH-SCORES 
FOR DIMENSIBN 6 - COOPERATIVE. 



(NSFORHED 






ii If ri ij w l\ 


M-SCORE 


29-29 


31-33 


3i»-36 3r-39 








mmmmm ••••• 


-100 


mm 


mm 


m m mm 


-99 








•98 








-9f 


mm 


mm 


mm 


-96 








-95 






• I 


-9^ 






• w 


-93 








-92 






• 0 


-91 








-90 








•89 






- Ti 


-88 








-87 


mm 


mm 


22 


-86 








-85 








-8(i 








-83 








-82 


29 


mm 


22 23 


-81 


29 


- 


23 23 


-80 


29 




23 23 


-79 


29 




2^ 2ii 


-78 


30 




2k 2k 


-77 


30 


mm 


25 2k 


•76 


30 


mm 


25 2k 


-75 


30 


23 


26 25 


•7* 


30 




2i 25 


-73 


30 


25 


26 25 


-72 


30 


25 


27 25 


•71 


30 


26 


27 25 


-70 


31 


27 


27 25 


-69 


31 


27 


27 26 


-68 


31 


27 


28 26 


•67 


31 


27 


28 27 


•66 


31 


28 


29 28 


•65 


31 


28 


29 26 


•6^ 


31 


28 


29 29 


-63 


31 


28 


30 29 


•62 


32 


28 


30 29 


-61 


32 


28 


30 30 



Hit 1 OF 5. 



\mP 












fO-i»2 






<»9-51 


52-5i» 


58-;s 
















IL 




4 V 


17 


m m 


Ik 


IL 


19 


Ik 


17 


mm 




IS 


13 


4 f. 


18 


mm 




16 




4 e 


18 


mm 




16 


15 


17 


19 


mm 


If 


&9 


Id 


4 i 

18 


19 


mm 


IQ 


17 
*f 


17 


4 A 


9fl 

cO 


mm 


10 


ll 


18 




9 A 

20 


mm 


2fl 
cv 


15 




cl 


20 


mm 


20 


18 


20 


21 


21 




2(1 


*7 


C\t 


99 


94 

Zl 




20 


IQ 


cu 


99 


94 

cl 




21 


19 






cl 




22 


20 


91 

44 


91 


97 

CO 




23 


21 


22 


23 


23 




24 




22 


91 


CH 




24 


21 




CH 


9i 

CH 




2(i 


22 


91 


9l 
CH 


9j 

CH 






22 




9li 

ch 


9j 

ci» 




25 


22 


2<» 


25 


2<l 




29 


23 


9c 


9C 


9e 

c5 


mm 


29 




2c: 


9t 
Co 


c5 


mm 


2^ 


2l 


9c 


CD 


9e 




2f; 


2l 


9c 


9t 

cr 


25 


26 


26 


2(1 


26 


27 


25 


27 


2ft 




9C 


99 

c7 


9e. 
CD 


27 


2A 

^0 


2e 


9C 
^D 


99 

c7 


9£ 

CD 


28 


27 


<b 


27 


9ft 

c8 


26 


29 


27 


2ft 


97 


9a 

c8 


9^ 

c6 


ID 


27 


26 


27 


28 


27 


30 


27 


27 


97 


9ft 

c8 


9* 

c8 


Jl 


27 


27 


9a 
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CO 
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c8 


Jl 




2a 
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CD 


9ft 

c8 


91 


2A 


2a 
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9q 
c^ 
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c8 


t4 
Jl 


28 


2t 


29 


20 
C7 


9ft 
CD 


19 

OC 


29 


29 


29 


29 


29 


32 


29 


29 


29 


29 


29 


32 


29 


29 


30 


30 


30 


32 


29 


30 


30 


30 


30 


32 


30 


30 


30 


30 


30 
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TABLE B-2. TABLE OF STANOARO SCORES FOR TRANSFORNEO RAN-SCORES 
FOR DIMENSION B - COOPERATIVE* 
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32 


28 


31 


30 
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30 
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30 


31 


30 
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30 


35 


-59 


32 


29 


31 


30 


30 


30 


31 


31 


30 


33 
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32 


29 


32 


30 


31 


31 


31 


31 


31 


33 
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32 


29 


32 


31 


31 


m s 

31 


31 


31 


31 


33 
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32 


29 


32 


31 


31 


31 
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32 


32 
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31 


33 
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32 


31 


32 


m A 

31 


31 
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31 


32 


32 


3Z 


mm 

33 
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33 


31 


32 


32 


32 


32 


32 


32 


m0k 

32 


34 
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33 


31 


32 


32 


32 


32 


32 


32 


32 


34 
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33 


31 


33 


32 


32 


32 
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33 


32 


32 


34 


-51 


33 


32 


33 


32 


33 


33 


mm 

33 


33 
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33 


34 
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33 


32 


33 


32 


33 


33 


33 


mm 

33 
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33 


34 
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33 


32 
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33 


34 


33 


34 


34 
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34 


34 


35 
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33 


34 


33 


34 


34 


34 


34 


34 
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h 
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34 


34 


34 


35 


34 
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34 


34 


35 
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34 


34 


34 


34 
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35 


35 


34 


35 
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34 


34 


34 
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34 


35 


m^ 

35 


35 


35 


35 


3> 
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34 


mm 

h 
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35 
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35 
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40 


40 
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40 
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40 
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^0 
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41 
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kl 
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41 
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HZ 
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41 
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kZ 
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42 


42 
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42 


42 
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-U 
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43 
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44 
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44 
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•5 


k7 


<>7 


47 


46 
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hi 


k7 
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46 


-S 


^8 


k% 




47 


-2 


^8 
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41 


-1 


%9 


49 
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4i 


0 


%9 
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49 


49 


1 


^9 


49 


50 


e8 


2 


SO 


SO 


50 
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SO 


51 
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so 


SI 


S2 
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S 


SI 


S2 


S2 


S3 


8 


SI 


SS 


52 


53 


7 


S2 


S3 


53 


54 


1 


S2 


Sk 


53 


54 

w^ 


9 


S3 


ss 


53 


55 


10 


S3 


56 


S4 


55 


11 


Sh 


56 


54 


56 


12 


9k 


57 


5S 


56 


13 


9k 


57 


ee 

WW 


56 


U 


Sk 


58 


56 


S7 
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57 


18 
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S8 


57 


SO 


17 


ss 


58 


57 


S8 


18 


S6 


S9 


58 


S8 


19 


S6 


59 


58 


$9 


21 


57 


60 


59 
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41 


41 


40 
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40 
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41 


41 
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41 
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Hi 


43 


43 


43 


42 


Ll 
Ht 


HI 


43 


43 


43 
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44 


43 


43 


43 
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H* 
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44 


44 


44 


43 


43 


42 


44 


44 


44 


44 
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45 


45 


45 
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HH 
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45 


45 


45 


H7 


H* 
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46 


46 


45 


45 


44 


46 


46 


46 


46 


45 


44 
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47 


47 


46 


46 


45 


48 


48 


4i 


L7 


Hr 


45 


48 


41 


4A 


HO 


HO 
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49 


49 


49 


H7 


H9 


46 


SO 


50 


50 


50 


SO 


49 


so 


SO 


50 


60 

7V 


el 

9* 
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SB 
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51 


si 


ei 


9C 


94 
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52 


52 


52 


63 


9€ 


52 


52 


52 


52 


63 
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9* 


52 


53 


53 


53 


53 


54 


S3 


53 


53 


53 

w^ 


6li 
9H 


99 


S3 


54 


54 

w^ 


64 

#H 


99 


99 


54 


54 

w^ 


94 


64 

*H 
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99 


99 


54 


55 


55 


66 

WW 
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99 


9r 


55 


55 


55 

WW 


55 


56 


57 


S5 


S6 


56 


66 


6& 
90 


90 


S6 


56 

Ww 


56 
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67 
9r 


St 
90 


56 


57 


Kf 


wf 


67 


SO 


57 


S7 


57 


67 

WW 
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90 
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99 


57 


50 


50 
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95 


95 


oB 


58 


SO 


SO 


so 


59 


&0 


SO 


S9 


59 


S9 


59 


61 


59 


S9 


59 


59 


59 


62 


59 


59 


60 


60 


60 
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TABLE B-2. TA9LE OF STANOARO SCORES FOR TRANSFORKEO RAN-SCORES 
FOR DIMENSION B - COOPERATIVE. 
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S7 


60 


S9 


60 


59 


68 


60 


60 
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\Z 


5t 


60 


60 


60 


60 
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60 
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60 
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\l 


60 


60 


61 


61 


60 


61 


61 


61 


61 
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Ik 


61 


61 


61 


61 


61 


61 


61 


62 


61 
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61 


61 


62 


61 


61 


62 


62 


62 
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62 


61 


62 


62 


62 


62 


62 


62 


62 
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62 


61 


62 


62 


62 


62 


62 


63 
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l9 
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61 


63 


63 


63 


63 


63 


63 
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63 


62 


63 
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63 


63 


63 


69 


63 
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hk 


63 


6^ 


63 


w 


63 


6^ 


6« 


Da 
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66 


63 


6^ 
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6^ 


6^ 


64 


63 
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12 


67 


63 


6$ 


6^ 


65 


6<» 
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6t» 
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IS 


67 


63 


69 


6^ 


65 


6^ 


65 
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\k 


68 


6I> 


63 
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65 


65 


65 


65 


64 


65 




19 


69 


6I» 


69 


69 


66 


65 


65 


65 


65 


66 




^6 


69 


6« 


66 


65 


66 


66 


66 


66 


65 


66 




1^ 


78 


6^ 


66 


69 


67 


66 


66 


66 


66 
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l8 


71 


6S 


66 


65 


67 


67 


66 


67 


66 
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71 


6S 
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66 


68 


67 


67 


67 


67 
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66 


67 


66 


68 


67 


67 


67 


67 
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66 


68 


66 


68 


67 


67 


67 


67 
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66 


69 


66 


68 


68 


67 


68 


68 
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67 


69 


67 


69 


68 


68 


68 


68 
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67 


69 


67 


69 


69 


68 


68 


68 
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mm 


67 


69 


67 


69 


69 


69 


69 


69 
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mm 


68 


69 


67 


70 


69 


69 


69 


69 


68 






mm 


68 


70 


68 


70 
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69 


69 


69 
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mm 


68 


78 


68 


70 


70 


70 


70 


69 
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mm 


69 


70 


69 


71 


70 


70 


70 


69 


69 






mm 


69 


70 


69 


71 


70 


78 


70 


70 
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mm 


70 


71 


69 


71 


71 


70 


71 
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71 


70 


71 


71 


71 


71 
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71 
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71 


71 
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73 
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6$ 
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7k 
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75 
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76 


77 
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77 
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mm 




77 
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7C 
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77 
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•* 


78 


76 


77 
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77 

ff 
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79 


77 


fO 


mm 
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71 
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mm 


79 


80 


79 


mm 


mm 


60 


81 


60 
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80 


81 
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mm 


mm 


a9 

OC 
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82 
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mm 
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83 


81 
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mm 
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86 


mm 


•m 


*• 


mm 
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7L 
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7C 
75 
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f^ 


75 


7L 

7h 


7C 

75 


7c 


75 


75 


7H 


mf 

7b 


7c 


7C 


75 




76 


n 
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7b 


75 


76 


7? 
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75 


mm 

76 


7c 
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7b 


79 
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77 


76 


7b 


77 


73 


77 
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77 
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77 
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77 

77 


7t 

7b 


77 


73 


77 


77 


7< 

7b 


77 


73 


77 


76 


7< 

7b 
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73 


78 


76 


7# 

7b 


77 


73 
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7A 


77 


77 
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7i 
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76 


77 

77 


77 
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7i 
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79 


76 


76 
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79 


78 


78 




79 


79 


78 


78 




73 


80 


78 


78 




79 


61 


76 
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mi 

01 


79 
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8c 


in 
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78 




81 


82 


60 


78 


mm 


19 


8Z 


61 


79 


mm 


8* 


8Z 


6c 


79 


mm 




8Z 


ii 

69 


79 


mm 




60 


83 


79 


mm 




83 




79 


mm 
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60 


mm 
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80 


mm 


09 


OH 
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80 
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69 


ik 


8d 


80 
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09 
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65 


65 
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81 


r 
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65 


67 
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mm 
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mm 
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mm 
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TABLE C-1. 



TABLE OF INITIAL, WM-SCORE TRANSFORATION COEFFICIENTS 
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9L 11)1} S^'S!! "'•^'^^ 112.595 ll7.(»8i» 128.377 131.199 

2J 75.i.3j 9*^.982 93.011 97.296 10(».000 108.666 112.983 117.889 128;819 131.650 

\l Jims VAW 5J''" "''•"^ "^•''»'» 129.230 132.069 

IS 'J'"' "'"^ 129.613 132.W0 

7 *1 lltl VAIl V'K^ 'M" « • " 132 21, 

IS tS* > 2!' S; 2?' ? "''•^oa 119.2W I30.33i» 133.166 

Z9 76.502 96*323 94.321 qa.&aS lflc.i.Ga na na hl cc. nm 130.618 133. (»86 

130 1913 133*. 786 

55 ^'^•^5* "''^^ "5.923 110.673 115.067 120.060 131.190 13(».070 
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Ifla Oil 


113.344 


U6.26S 


75.905 
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96*750 


94.738 
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A*w.0^7 
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95.109 
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mmm 
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70 


70 


57 


mm 


75 


73 


70 


n 


71 


70 


7t 


70 


58 


mm 


77 


7i» 


71 


72 


71 


70 


71 


70 


59 


mm 


•* 


7^ 


71 


72 


72 


71 


71 


70 


60 


mm 


mm 


75 


71 


72 


72 


71 


71 


70 
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29-29 


31-33 




37-39 




it3-t»5 


i»5-i»8 


49-51 


52-54 




61 


« • • • • 
« m 


• • 


« • • 

75 


72 


• • mmm 

73 


72 


• • m 

71 


71 


71 


70 


62 


mm 


mm 


75 


72 


73 


72 


72 


72 


71 


71 


63 


mm 


mm 


79 


73 






72 


72 


71 


72 


OH 






77 




7<» 


73 


72 


72 


71 


72 


6S 


mm 


mm 


77 


73 


7«» 


73 


72 


72 


71 


75 


66 


mm 


mm 


76 


7t 


7'» 


7^ 


73 


72 


72 


73 


67 


mm 


mm 


71 


7k 


75 


7^ 


73 


73 


72 


7'4 


6S 


mm 


mm 


79 


75 


76 




73 


73 


72 




Of 






7Q 

f 7 


f 7 


75 


75 


1 H 


73 


72 


mm 


70 


mm 


mm 


60 


76 


77 


75 


7h 


7k 


73 


mm 


71 


mm 


mm 


B3 


76 


77 


76 


75 


7«» 


73 


mm 


72 








77 


77 


75 


75 


7'» 


73 


mm 


73 


mm 


mm 




76 


7A 
f 0 


f 0 


76 


75 


74 


mm 


71. 








7 A 

r 0 


76 


77 


f 0 


7Q 


74 


mm 


75 








79 


79 


77 


77 


75 


74 


mm 


76 








60 


60 


76 


77 


76 


75 


mm 


77 


mm 


mm 


mm 


60 


61 


79 


76 


76 


75 


mm 


7« 








61 


Ml 
Oi 


7Q 
f 7 


76 


76 


76 


mm 












63 


79 


r 0 


f 0 


7fc 


mm 


ao 










ik 


6*0 


79 


76 


76 


mm 


61 










65 


80 


79 


77 


76 


mm 


62 


mm 


mm 


mm 


mm 


65 


61 


79 


77 


77 


mm 


63 
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61 


76 


76 


mm 














62 


0* 


74 


7Q 


mm 


6$ 












62 


61 


79 


79 


mm 


66 












62 


62 


79 


89 


mm 


67 












8V 


62 


60 


60 


mm 


68 














83 


60 


60 


mm 














67 


S3 


81 


80 


mm 


90 












66 


65 


62 


81 


mm 


91 














86 


62 


61 


mm 


92 














66 


63 


61 


mm 


93 














66 


ik 


81 


mm 


9V 














89 


6<i 


62 


mm 


9S 
















85 


82 


mm 


96 
















66 


62 


mm 


97 
















66 


82 


1 


96 
















87 


83 


• • 


99 


















63 




100 


















83 


• • 


•••••• 
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• •••• 


• • 






• •••• 
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